Search for: All records

How do we make a machine that indicates changes to its internal state, e.g., status, goals, attitude, or even emotion, through changes in movement profiles? This workshop will pose a possible direction toward such ends that leverages movement notation as a source for clearly defining abstract concepts of similarity and symbolic representation of the parts and patterns of movement - in order to identify, record and interpret patterns of human movement on both the micro and macro levels. First, we will move together. This will activate an innate ability to imitate each other and, in doing so, illuminate the principal components of Laban/Bartenieff Movement Studies – a field comprised of Laban Movement Analysis and Bartenieff Fundamentals – and the Body, Effort, Shape, Space, and Time (BESST) System of movement analysis. This system of work, deriving from dance and physical therapy practices, which is a textbook; thus, a key value proposition of the workshop is in its embodied, situated nature that can be supplemented by textbooks, including a newly released book from MIT Press authored by the workshop organizers. Next, we will try to write down what we’re doing. A set of symbols for describing elements of the BESST System, which seem to be particularly perceptually meaningful to human observers, will be presented so that movement ideas can be notated and, thus, translated between bodies. We will explore both Labanotation and a related “motif”-style notation. This workshop is supported by NSF grant numbers 2234195 and 2234197. 

The British landscape painter John Constable is considered foundational for the Realist movement in 19th-century European painting. Constable’s painted skies, in particular, were seen as remarkably accurate by his contemporaries, an impression shared by many viewers today. Yet, assessing the accuracy of realist paintings like Constable’s is subjective or intuitive, even for professional art historians, making it difficult to say with certainty what set Constable’s skies apart from those of his contemporaries. Our goal is to contribute to a more objective understanding of Constable’s realism. We propose a new machine-learning-based paradigm for studying pictorial realism in an explainable way. Our framework assesses realism by measuring the similarity between clouds painted by artists noted for their skies, like Constable, and photographs of clouds. The experimental results of cloud classification show that Constable approximates more consistently than his contemporaries the formal features of actual clouds in his paintings. The study, as a novel interdisciplinary approach that combines computer vision and machine learning, meteorology, and art history, is a springboard for broader and deeper analyses of pictorial realism. 

Bodily expressed emotion understanding (BEEU) aims to automatically recognize human emotional expressions from body movements. Psychological research has demonstrated that people often move using specific motor elements to convey emotions. This work takes three steps to integrate human motor elements to study BEEU. First, we introduce BoME (body motor elements), a highly precise dataset for human motor elements. Second, we apply baseline models to estimate these elements on BoME, showing that deep learning methods are capable of learning effective representations of human movement. Finally, we propose a dual-source solution to enhance the BEEU model with the BoME dataset, which trains with both motor element and emotion labels and simultaneously produces predictions for both. Through experiments on the BoLD in-the-wild emotion understanding benchmark, we showcase the significant benefit of our approach. These results may inspire further research utilizing human motor elements for emotion understanding and mental health analysis. 

Emerging computing paradigms provide field‐level service responses for users, for example, edge computing, fog computing, and MEC. Edge virtualization technologies represented by Docker can provide a platform‐independent, low‐resource‐consumption operating environment for edge service. The image‐pulling time of Docker is a crucial factor affecting the start‐up speed of edge services. The layer reuse mechanism of native Docker cannot fully utilize the duplicate data of node local images. In this paper, we propose a chunk reuse mechanism (CRM), which effectively targets node‐local duplicate data during container updates and reduces the volume of data transmission required for image building. We orchestrate the CRM process for cloud and remote‐cloud nodes to ensure that the resource overhead from container update data preparation and image reconstruction is within an acceptable range. The experimental results show that the CRM proposed in this paper can effectively utilize the node local duplicate data in the synchronous update of containers in multiple nodes, reduce the volume of data transmission, and significantly improve container update efficiency.

 

As inborn characteristics, humans possess the ability to judge visual aesthetics, feel the emotions from the environment, and comprehend others’ emotional expressions. Many exciting applications become possible if robots or computers can be empowered with similar capabilities. Modeling aesthetics, evoked emotions, and emotional expressions automatically in unconstrained situations, however, is daunting due to the lack of a full understanding of the relationship between low-level visual content and high-level aesthetics or emotional expressions. With the growing availability of data, it is possible to tackle these problems using machine learning and statistical modeling approaches. In the talk, I provide an overview of our research in the last two decades on data-driven analyses of visual artworks and digital visual content for modeling aesthetics and emotions. First, I discuss our analyses of styles in visual artworks. Art historians have long observed the highly characteristic brushstroke styles of Vincent van Gogh and have relied on discerning these styles for authenticating and dating his works. In our work, we compared van Gogh with his contemporaries by statistically analyzing a massive set of automatically extracted brushstrokes. A novel extraction method is developed by exploiting an integration of edge detection and clustering-based segmentation. Evidence substantiates that van Gogh’s brushstrokes are strongly rhythmic. Next, I describe an effort to model the aesthetic and emotional characteristics in visual contents such as photographs. By taking a data-driven approach, using the Internet as the data source, we show that computers can be trained to recognize various characteristics that are highly relevant to aesthetics and emotions. Future computer systems equipped with such capabilities are expected to help millions of users in unimagined ways. Finally, I highlight our research on automated recognition of bodily expression of emotion. We propose a scalable and reliable crowdsourcing approach for collecting in-the-wild perceived emotion data for computers to learn to recognize the body language of humans. Comprehensive statistical analysis revealed many interesting insights from the dataset. A system to model the emotional expressions based on bodily movements, named ARBEE (Automated Recognition of Bodily Expression of Emotion), has also been developed and evaluated. 

Developing computational methods for bodily expressed emotion understanding can benet from knowledge and approaches of multiple fields, including computer vision, robotics, psychology/psychiatry, graphics, data mining, machine learning, and movement analysis. The panel, consisting of active researchers in some closely related fields, attempts to open a discussion on the future of this new and exciting research area. This paper documents the opinions expressed by the individual panelists. 

Body orientation estimation provides crucial visual cues in many applications, including robotics and autonomous driving. It is particularly desirable when 3-D pose estimation is difficult to infer due to poor image resolution, occlusion, or indistinguishable body parts. We present COCO-MEBOW (Monocular Estimation of Body Orientation in the Wild), a new large-scale dataset for orientation estimation from a single in-the-wild image. The body-orientation labels for around 130K human bodies within 55K images from the COCO dataset have been collected using an efficient and high-precision annotation pipeline. We also validated the benefits of the dataset. First, we show that our dataset can substantially improve the performance and the robustness of a human body orientation estimation model, the development of which was previously limited by the scale and diversity of the available training data. Additionally, we present a novel triple-source solution for 3-D human pose estimation, where 3-D pose labels, 2-D pose labels, and our body-orientation labels are all used in joint training. Our model significantly outperforms state-of-the-art dual-source solutions for monocular 3-D human pose estimation, where training only uses 3-D pose labels and 2-D pose labels. This substantiates an important advantage of MEBOW for 3-D human pose estimation, which is particularly appealing because the per-instance labeling cost for body orientations is far less than that for 3-D poses. The work demonstrates high potential of MEBOW in addressing real-world challenges involving understanding human behaviors. Further information of this work is available at https://chenyanwu.github.io/MEBOW/ .